DOCUMENT RESUME 

ED 316 190 IR 014 177 



AUTHOR 
TITLE 



INSTITUTION 

REPORT NO 
PUB DATE 
NOTE 

PUB TYPE 



Bestebreurt je, R, 

Report on Participating in the DELTA Pre-Pilot 
Project: Building a Reference Model for European 
Learning Systems. 

Twente Univ., Enschede (Netherlands). Dept. of 

Education. 

ISBN-90-365-0285-3 

Jun 89 

50p. 

Viewpoints (120) — Reports - Research/Technical 
(143) 



EDRS PRICE 
DESCRIPTORS 

IDENTIFIERS 



MF01/PC02 Plus Postage. 

^Communications; ^^Educational Technology; Foreign 
Countries; International Programs; 5»cModels 
^Europe 



ABSTRACT 
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as a guide to critical processes, and as a reference to critical 
issues are assessed in the fourth section. The final section offers 
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FOREWORD 



The DELTA Pre-Pilot served many important functions. Some, of course, 
related to its Immediate output (for example, over 1 10 reports were for- 
mally submitted). However, many of the valuable lessons learned during 
the Pre-Pilot activity will have application in the future as much as or even 
more than in the present. It is inevitable in this future that demands for co- 
operation a^nong Europeans of different professional backgrounds and 
mother tongues for the development of technology-supported learnir>g 
systems will escalate. The DELTA Pre-Pitot gave partners from ten such 
diverse groups the opportunity to experience the potential in such cooper- 
ation but also to better anticipate the problems. 

And problems there are. Communication, even amor^g professionals with 
common work experiences, becomes more challenging as complexity of 
concept and terminology becomes more important. Standardization of 
communication and interactton procedures become as important as stan- 
dardization of hardware and software environments if productivity and 
common purpci^is are to be maintained. The DELTA Pre-Pilot gave us the 
opportunity to contribute to a better level of communicatton through the 
vehicle of the so-called "cubic framework" of a learning systems reference 
model (LSRM) which the Pre-Pilot evolved. 

Bestebreurtjci, the author of this report, is well qualified to write such a 
summary. Hd was an active member of the University of Twente team in 
the Pre-Pilot and was closely involved in the preparation of several key re- 
ports as well as in the evaluation of the cubic framework. In his summary 
he clarifies not only the progress of the Pre-Pilot from the perspective of a 
participant b Jt also identifies a number of key points for further devebp- 
ment in the urea of European learning systems. Thus this summary is not 
only on Insightful description and analysis but also has strong proscriptivp. 
value, There are various members of the DELTA Pre-Pilot team from the 



University of Twente who are currently Involved in the next stage of 
DELTA; among these are Bestebreurtje, Moonen, and myself. The experi- 
ences gained while participating in the DELTA Pre-Pilot, as summarized 
by this report, gives us a valuable entry point to the new project. 



Betty Collis v 

University of Twente 

Department of Education 

Division of Educational Instrumentation 

PO Box 217 

7500 AE ENSCHEDE 

The Netherlands 



30 May 1989 
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SUMMARY 



1. Inlfodudion. 

Th9 main purpose of the DELTA*'Pre-Pilot project (Development of 
European Learning through Technological Advance) was to develop a 
European Learning System Reference Model (LSRM). This LSRM serves 
the purpose of bringing structure In the area of European Learning 
Technology in order to function as a kind of Intermediary and consensus- 
Identifying tool. 



Activities. 

in thi;<^ paper a description is given of various activities occurring within the 
Pre-Pilot and how these activities generated a "Cubic Frameworks 



3. The ModeL 

The collaboration produced a framework represented by a three- 
dimensional cube which has as dimensions: 

ACTIVITIES / ACTORS / RESOURCES 

Each dinr ension exists of major terms which are subdivided. So the cube 
has 15 X 13 X 4 « 780 main-cells, each of which can be described in the 
context of: 

•"Whatisdone / by whom / by what moans'* 



Applications of tho Modol. 



The framework is seen to have at least three areas of aipplication. In this 
paper all three areas are illustrated. 

The first application is for communication purposes. Tho description of 
each cell's subdivision is in standard('*iz6d) terms to facilitate 
communication throughout Europe. 

Another service of the framework is as a gukie to critical processes 
occurring within some aspect of development, distributton, or use of 
learning materials. An example is given of how the framc^work can serve 
as a guide to a set of critteal processes by taking a particular "slkse* of the 
"Cubic Framework** bek)nging to the term "Specity* (of the general term 
"Develop**) and ordering its **activitiGs** and their specif k: sub-cells using a 
flow-chart approach. This example demonstrates both tho use and the 
limitations of the guide-function. 

Finally within the DELTA Pre-Pik)t project some areas of critical problems 
surrounding the development and use of learning materials on a Europe- 
wide basis were explored. The flow-chart approach providos a mechanism 
by which the cubic framework can serve as a reference to these critical 
issues at stages in which they are (or seem to be) relevant. Outcomes of 
these investigations are discussed. 



Further Research. 

In future this model is to be elaborated in the next phase of DELTA (the 
**Exploratory Action**). There is a specific Action JLine (AL. 1) dt^signed as 
the place for harmonizatbn and integration of all the research done within 
the other Action Lines. One component is the refinement and frjrther 
development of the LSRM, as built upon the **Cubic Framework", as a 
setting for the research in the DELTA area. Another component is the 
foundation of a European Learning Technology Association (ELTA) as the 
place for exchange of information. 
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1; INTRODUCTION. 



Background. 

"Learning** as it is seen in the initial Studies on DELTA, is an Important 
information gathering activity and a major basis for the ability to solve 
problems within a society and to shape its future. In the modern western 
societies Information Technology - all methods, techniques arnl products 
stemming from computer science and related disciplines is a major 
stimulus of essential social, economical and cultural changes. Within the 
European Communities the European Commission desires to be a catalyst 
for cooperative activities In the area of Learning and Information 
Technology. This Is mainly based on the assumption that In the early 21^^ 
century the average European wH| have to be retrained about four times in 
her/his 'active life* and on the l^nowledge that the European countries have 
excellent educational systems and a high standing In Learning Technology 

- Information Technology applied In the area of Learning (see Initial 
Studies. 1987, p. XXXII). 

The general idea is that if, on the one side, there Is a growing 
("accelerating") need for retraining and development of specific courses In 
spe:lallstic areas and on the other side, there Is a small -but also growing- 

- Infrastructure of advanced learning, use can be made of the large market 
pos sibllities that the European Communities can offer. This can lead to 
COS' reduction and efficiency In often small, but fast changing specialist 
areas (one does not have to Invent the wheel again). 
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1 .2. The DELTA Programmo 

In order to achieve both the development of a ''European infrastructure on 
Advanced Learning" (i.e. the area of Learning and Information 
Technologies) and more standardization in the development of courses in 
specialist areas* the Commission of the European Communities initiated 
the DELTA programme (Development of European Learning through 
Technological Advance) in 1987. This project is intt^nded for a duration of 
about ten years. 

The DELTA programme aims at developing and establishing an European 
capability in distance learning and supporting communications 
infrastructure. The "Pre-Pilot project** was a preliminary phase to the 
DELTA programme. The main purpose of this Pre-Pilot project was to 
devetop a ''European Learning System Reference Model" (LSRM). The 
LSRM serves the purpose of bringing structure into dis^cussions within the 
area of European Learning Technology and of functioning as a Idnd of 
intermediary and consensus-identifying tool. 



1 .3. The organization of the DELTA Pre-Pllot project. 

In the DELTA Pre-Pilot project ten organizations representing industry, 
publishers and academic institutbns from five European countries worked 
together in a consortium during 1988 In the contract for the Pre-Pilot, 
eleven tasks were described anH grouped together in four Task-Forces". 
Each task would be ended with a summarizing Final Report on that task. 



1 ) The participants in the DELTA Pre-Pilot project are: 
-Bull (F): 

-KlettVeriag(FRG): 
•.MacMillanlntdU(UK); 
-Olivetti (I); 

-Open University (UK); 
-Philips (NL); 
-CEN/Saclay (F); 
-SESA(F); 
-SIDAC(I); 



University of Twente (NL). 
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Besides that the European Commission required two Intermediary Reports 
during 1988 and a Final Report at the end of the Pre^Pilot project. The 
execution of the eleven taslis was to first evolve and then assess a 
**Learning System Reference Moder (LSRM). 



Atsumptlons. 

Three general assumptions lay behind the DELTA Pre-Pilot project: 

1 . The general idea behind modeling the "DELTA area** in the Initiating 
literature of the European Commission was that there should one 
''shell-system'* In Europe (e.g. callecf '*DELTA**), that mediates all 
demands and supplies in the ar«A of "^advanceo iesrnlng**. Strategies 
for tnis mediation refer to the application of advanced technologies. 
This 8hell*system should be theoretical!^' grounded on the basis of the 
LSRM. However the primary quistlon, if the need and the will for this 
kind of centralizing exists, was not posed. 

2. A second assumption was that there was already a definition of such a 
model (refer to: Initial Studies, 1987). However, during the project this 
assumptton emerged as being wrong: there was no clear definition of 
an LSRM or even a common concept for it. 

3. This second assumption was based upon another, nam^ttv that 
generally standardized practices in producing and using advanced 
learning materials were already existing, tt was thought that out of 
these practices the model could be generated easily. He vever, there 
was little consensus about practice or on a development methodology 
which could be generalized* So halfway through the Pre-Pilot project 
its workplan was rescheduled and refocused i i developing some kind 
of framework towards an LSRM. 



In this paper this development process, the outcome, the results (a "Cubic 
Framework**) and its assessment will be discussed Also some remarks will 
be made about Input for the next phase of DELTA research. 



2: ACTIVITIES. 



2.1. Introduction. 

As mentbned before, the Pro-Pilot project of DELTA was divided into 
eleven tasks, which in turn were grouped in four Task Forces covering the 
major aspects of this Pre-Pibt. 

2.1 .1 . An overview of the task f orcee. 

Task Force t was meant to define the sample for the further activities of the 
DELTA Pre-Pilot project with regard to the setting of these activities, as 
was written in the Initial Studies of DELTA (refer to: Initial Studies, 1987). 

In the Task Forces tt and III specific issues were exptored. This was to 
shape the context for a second thrust of work for Task Force I, the task of 
further evolving a **European Learning System Reference Moder (LSRM). 
Finalty Task Force IV was to prototype some of these findings in the 
context of the LSRM and provkie input to the overall Task Force I, so that 
Task Force I could complete its third general activity, the assessment of 
the LSRM in the context of the chosen prototyping aspects. 

In this section a description of the outcomes of the context-shaping task- 
forces (II and III) will be the major focus. 
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2.2. Task Forces within ths Prs-Pllot. 



2.2.1. Task Force I: Definition, Selection and Validation. 

The major purpose of Task Force I was to define and select a sample field 
of learning and a subset of a target audience, Including their assumed 
prerequisite knowledge, for the prototyping activities of Task Force IV. Also 
the validation of the prototyping at the end of the project was part of the 
assessment of the LSRM which was also the responsibility of Task Force L 

Oiitcomes of Task Force I will be presented In Sections 3 and 4 of this 
paper. 

2.2.2. Task Force II: Standards and Interoperability. 

In this Task Force, in which we (University of Twente) were not Involved, 
several aspects were considered. Bebw they are briefly described. 

The consklered aspects were related to the current and future state of 
market standards -possible media mixes, technological trends, and 
general functional requirements- and to alternative options for the delivery 
of learning material (learning support systems). 

Outcomes of the studies within Task Force II are: 

A: About standardization: 

A plea for system Interoperability Including: 
Hardware compatibility. 

CD-I (closed system) or an MS-DOS PC (open system). 
Systems configuration, to Increase the likelihood of portability. 
Basic Intelligent Workstation and enhancements (CD-ROM/CD-I, 
vok^e treatment, videodisc player 4* graphic overlay adapter. 
Presentation/Interaction. 

Ease for use of unskilled users and uniformity In the user-Interfaces. 
(Note: OS/2 Presentation Manager Is assumed to soon be a 
standardized approach to the user interlace across different classes 
of systems, from MS-Dos to Unix.) 
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open access to resources and systems interconnections. 
The interconnection of differer^t systems impiies the choice of 
proiocol standards. Ihe ISO\OSI iayers (Internationai Standards 
Organization/Open Systems Interconnection) couid be that standard. 

B: About Learning support systems: 

Three different educationai situationo were distinguished. They are not m 
utuaiiy exciusive: in reaiity we can see combinations of two or aii three of 
them. These three theoreticai situations can be described in the foiiowing 
modeis: 

Self-instruction model. 

The tutor is embedded in the technoiogy-based system. An adequate 
system could substitute for the human tutor. 
Local area model. 

In a local area environment there is interaction among learners and 
between a learner and a (human) tutor, as well as access to local 
information resources. 
Distance interaction nnodel. 

There is remote interaction among learners and between a learner 
and a (human) tutor, as well as access to (distant) information 
resources. 

These three models are not exclusive to computer assisted learning. 
However, when applied in a CAL**setting, cx>nsiderdtion should be 
tal<en of the issues summarized under A: Standardization. (Refer 
also to: Final Report, 1989, Chapter 4.3.). 



2.2.3. Taek Force III: Educational requirements 

In this Task Force elements important for the core activities of developing 
learning materials (mainly: ''specify^ *'design^ See also Section 3, below) 
were considered. 
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Major conclusions from this Task Force were: 

A: After defining the precise learning objectives relative to the target 
population and to make these objectives explicit In measurable 
terms, one has to take account of the specific requirements of the 
course that Is needed. The following eight types of courses are 
distinguished: 

General Interest courses, assuming no prior expertise; 

Training and re-tralnlng courses, to improve skills (practical or 

intellectual); 

Up-dating courses, to teach addlttonal knowledge to already 
qualified specialists; 

Awareness courses, to teach krx>wledge and understanding 
(Instead of skills) at a fairly superficial level to specialists In 
other fields; 

Undergraduate courses, to teach knowledge, skills and under- 
standing for the purpose of gaining a first degree; 
Up-grading courses, to obtain higher (vocattonal) qualifications; 
Converslo^i courses, to assist people who are (highly) qualified 
in one f leki as they move into another; 
Post-graduate courses, similar to up-grading but mainly 
focused on gaining a higher degree, instead of being 
vocattonally oriented. 

The use of this categorlzatton can lead to selector of related 
pedagogteal strategies by the authors of a ooursf and to more 
effective focusing on the selectton criteria on btinalf of the 
learner. 

B: The selection of pedagogical strategies depends on the type of 
course that Is required (see above) and the kind of learning situation 
that will be ^'used'* (see TF.II, above). Parallel to these, aspects such 
as the presumed skills and understandings of the Intended learner 
should be taken Into account and made explicit. Other considerations 
In this stage of development of courseware Include how to anticipate 
different learning styles and how to build In assessment of the 
learner's progress In order to optimally reach the learning objectives. 

C: To anticipate the European context, which finally should be the major 
scope of the DELTA programme, the following general notions were 

also staled: 
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There Is a need for a well-organized Information base on tfie 
availability of various kinds of learning materials throughout the 
European Community. In the context of oosts-reduction this 
material could also be available for re-use (translation, transfer 
or adaptation) 

There is still a lack of compatibility of equipment standards. It is 
especially in the European interest to olve these problems. 
The way towards a "European Learning Market" is only via their 
solution. (Suggestions were given in the description of Task 
Force II, above.) 

National differences are perceivi J in the European countries, 
because of separate development of their educatkinal 
traditions. This can be coped with by antk:ipating specifk: 
learning styles and by actively using a standard set of icons, 
whk:h has still to be devetoped. Another option is to devek)p 
courseware that has separate text, pk^oriai and structural 
elements. The translatkin (whk:h is only one kind of re-using) of 
courseware could then be focused mainly on the text elements. 



2.2.4 Task Force IV: Authoring and prototyping. 



The prototyping activity was set up to address some of the specifications 
made by the DELTA Pre-Pitot studies and to assess the "LSRM" as it was 
evolved in Task Force I. However, because of the rescheduling of the 
DELTA Pre-Pilot project the actual activities of this task force fall outside 
the scope of this paper. 
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Romarks. 



Some remarks can be made: 

As noted in the previous section, the assumption that there was already a 
kind of an LSRM appeared to be not true. This implied that Task Force I 
could not further evolve that model. When starting the prototyping activity 
the lack of agreement about a kind of model became clear. At that time, 
halfway through the project, Task Forces I and IV were reorganized: Task 
Force IV shouW, in their prototyping, seek to validate selected issues aris- 
ing from the work of the other two task forces and at the same time Task 
Force I shoukl outline a first framework for an LSRM, whk:h the 
prototypers should use when reporting on their findings. The general kfea 
at that time was that In this way the DELTA Pre-Pik)t project could still do 
both of the foltowing things: 

- deal with some major issues and 

- build and assess a Learning System Reference Model. 
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3: THE MODEL 



Introduction. 

The major goal of the DELTA Pre-Pilot project was to (further) develop 
Ideas about a **European Learning System Re/erence Model" as a kind of 
Intermediary and consensus-identlfying tool. 

The first Idea (see e.g. Delta Initial Studies, 1987) was to articulate one 
central system (an "LSRM") within which suppliers and users of learning 
materials could communicate. This system-to-be-developed would Involve 
a number of restrictive factors that should be considered: 

- Suitability for centrally stored Informatbn or centrally placed tools; 

- Cost and speed of communications, especially if use Is made of 
conventional broadcasting media, audio and video cassette; 

- TImescale of usage growth and possibilities of duplicating the databases 
in member countries at reasonable distribution rates and acceptable 
costs as well as setting up the necessary organization to run such 
facilities; 

- Possibility of offering free communication outside *t>usy traffic" hours. 

This concept was worked out In the DELTA Pre-Pibt project, among oth- 
ers, in a paper called "Towards an LSRM" within the context of Task-Force 
III Its essence was that the model It described, was a blue print for one 
system In which a combinatton of distance learning and In-company 
(corporate) training was devetoped. The role of the DELTA Pre-Pllot pro- 
ject was seen in that paper as being of help to clarify the objectives, 
structure and possible modes of operation of such a European Learning 
System Reference Model. Beyond that the need for research, 
specification, and development of standardized resources (tools) and 
interfaces required to bring the system Into existence was felt to be the 
most important task of the (10 years duration of the) DELTA programme. 
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Evolution of a "Cubic FranMWork". 

At about halfway through the project, the participants who maintained their 
focus on modelling the area of European Learning to continue the building 
of an LSRM made the choice to take current (European) learning-material- 
development environments as a point of departure for description and to 
focus upon certain aspects, such as which actors are involved in which ac- 
MifiS and what tools do they use. This implied a choice for a three-di- 
mensional description, that in turn led to the construction of a "cube". 

Four examples of production strategies for multi-media learning materials 
in Europe were described In terms of "who does what at whk:h stage of the 
development and which tools/resources are being used". These examples 
were In the U.K.: Open University and Macmillan Intek and In the Nether- 
lands: the POCO-projoct and an experimental authoring environment, 
called EDUC, of the University of Twente, Department of Instrumentation 
Technology). 

After these descriptions generalizations about development and production 
environments were made by the coordinating team of Task Force I. The 
results of these generalizations were then used to re-describe the chosen 
environments in those general terms In order to check the applicability of 
the terms. The resutt of this exercise was a "cubic framework" with as 
dimensions "Activities", "Actors" and "Resources", with standardized 
descriptors on each dimension, 

In the following overview (for the full developments, see the Final Roport, 
1989, Chapter 2) gives the major terms per each dln^ jnslon involved In the 
"Cubic Framework" (see Appendix A for more details): 
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Dimensions ; 



Major terms : 



Activities: -Manage 

-Facilitate; 
-Standardize; 
-Supply; 
-Develop 



-Distribute; 
-Use; 
-Evaluate. 

Actors: -Government; 

-Society; 
-Expert/Academic body; 
-•Manufacturer; 

-Developer (incl.: Project Management, 
Specification staff, Design 
staff, Production staff, 
Assembling staff. 
Evaluation staff, Servlcas 
staff); 

-Publisher/Servicer; 

-Client. 

Resources: -Human Skills & Understanding; 

-Methodologies; 
-Software; 
-Hardware. 



(incl.: General, Marketing, 
Sales, Support); 



(incl.: Specify, Design, 
Produce, Evaluate, 
Assemble); 



Ftgurs 1 : Tha cubic Framtwork. 
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RESOURCES 



ACTIVITY 



1 



ACTOR 



MANAGE: 

« tuppon 
FACILfTATE 
SUPPLY 
DEVELOP: 



•produoi 

DISTRIBUTE 
USE 

EVALUATE 



Final Report, 1989, p. 23 
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During the development of this standardized "cube** as a framework for the 
DELTA LSRM, the following three characteristics emerged as being 
particularly important: 

1 . The DELTA LSRM should be an adequate communication tool, pro- 
viding a general descriptive framework which can be applied to differ- 
ent European situations involving the development and distributton of 
learning materials. 

2. The DELTA LSRM should provkJe a gukie for crltk:al processes within 
learning materials devebpment and distribution. 

3. The DELTA LSRM should be able to serve as a reference tool relative 
to critical issues and problems related to learning materials devetop- 
ment and use. 

(Final Report, 1989. p.153.). 



A specific task in tiie DELTA Pre-Pilot project was to assess the devetoped 
LSRM. The "Cubic Framework" was assessed according these three 
characteristics. The results of this -limited- assessment were optimistic 
(see Appendix K of the Final Report, 1989) and they will be described in 
the next section. However, after finishing the project, some observations 
can be made including reviewing the factors that were presented in the in- 
troduction of this section that should have been considered . 

- An important change in the way of thinking can be noticed. The specifk: 
example of this change can be noted in the change in the way "LSRM"* Is 
spelled since the reset of the Task Forces halfway the DELTA Pre-Pilot 
project: Learning System* Reference Model. This implies that the original 
notion of constructing a central model as a blueprint for a real system-to- 
be-developed, as was refered in the Initial Studies, 1987, no tonger ex- 
isted. So the first assumption underlying DELTA was dismissed and the 
restrictive factors (see Section 3.1.) were out of consideratton. 

The concluston appears therefore, that the DELTA Pre-Pilot project has 

shaped its own criteria for a review. 
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Other general aspects to be mentioneci here ar^: 

- The area of description is extremely large. 8y trying to incorportate 
everything the model becomes meaningless. For example there can be a 
legislator who defines a general curriculum for a specific institute of 
education at the one side and at the other side there can be the 
Telecommunications operator who is involved in the technical realization 
ot a connection with a specific network. The question is how fundamental 
are their contributions in terms of a useful European model on Learning 
Technology); 

- The subdivisioa (see Appendix A) is very complicated because of the ex- 
tremely large range of possibilities; however.from another perspective 
little definitbn of the terms has been made. This makes the model very 
oomplk»ted. Crossing all the dimenstons at this subdivkJed level leads to 
73,602 cells. The Final Report says about the Cube: "... (the Framework) 
appears to be a manageable instrument for structuring the complex 
environment in whk:h DELTA is operating. It Is a major step forward that 
all the project members agreed on this Framework." (Final Report, 1989, 
p.26.). The question remains however, if the Cubk: Framework is still 
more than the sum of its construing parts, the dimenstons. 

The concluston of this section is that the DELTA Pre-Pik>t project has de- 
vek)ped differently than its original purposes. However, although its deliv- 
ery is not a system, the "Cubic Framework" at least can be a first step to- 
wards a more generally accepted methodok>gy if it is able to function ac- 
cording the three characteristk» mentk^ned above. This final aspect will be 
conskiered more ctosely in the next section. 
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4: ASSESSMENT OF APPLICATIONS OF THE MODEL 



introduction. 



As is stated in the previous section the application of the ''Cubic Frame* 
worli" should serve three purposes: 

1 . to be an adequate oommunication tool; 

2. to provide a guide to critical processes; 

3. to serve as a reference to critical issues. 

In this section each of these three aspects will be discussed. Al^ some 
conditions are stated about the output of the DELTA Pre-Pilot project rela- 
tive to the Expb^atory Action as an Introduction to the next sectbn. 



The *'Cublc Framework** as a tool for communication. 



The assessment of this aspect of the "Cubic Framework" was mainly done 
by redescribing the four considered "environments'* of producing course- 
ware '(the Open Univerisity, Macmillan Intek, POCO and EDUC) in terms 
of the "Cube". (For a detailed overview of this exercise refer to the Final 
Report. 1989. Chapters.) 

Some aspects about this exercise can oe mentioned: 

- Within the context of all the four of the described environments, the termi- 
nology of the "Cubic Framework" could clearly be applied; however it 
should be noted that these four descriptions also grounded the "Cube" in 
that they contributed substantially to its development; 

- The three dimensions were mainly considered separately, so the discus- 
sion was not focused upon the cells but on the individual terms; 
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• Thera were •normous difffirancas among tha four oonsiderad environ- 
monts. Thtt most strikirtg diff«renc« was that three examples ("POCO". 
the "Open University" and "MacMillan lntel<") described general 
approaches to courseware development, and were more or less focused 
on management aspects, while the other ("EOUC") descritMd the more 
limited, but more specific aspects of the actual specifying, programming 
and testing of courseware. 

It is obvious that the "Cube" has the potential to function as a tool for com- 
munication purposes. Itewever. four remarl<s should be made: 

1 . The terms still need a precise definition. Within the DELTA Pre-Pilot 
project they were used with a "common understanding" but outside the 
project it should be clearly specified what Is meant by oach term (and 
each cell). 

2. To become an general, accepted basis for standard terminology the 
"Cube" still needs to be extended, completed and refined relat^;e to 
other environments for courseware development. 

3. To remain usable and accessible, the 'Cubic Framework" needs to be 
implemented in something lil<e a hypertext-system, which could still be 
a substantial step towards the original purposes of setting up a 
DELTA-system (see also Section 3). 

4. It will be useful if a selection of ant>y point or perspective can be made 
according the "type" of environment that is considered before access- 
ing the "Cubic Framework\ 

These remarlts should be considered in the next phase of DELTA. How- 
ever, in the context of the DELTA Pre-Pilot project specific suggestions 
were made for an extension of the last two rtmaiks relative to the use of 
an hypertext approach and to selecting an entry point according the level 
of organization, (see also Section 5) 
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4.3. The "Cubic Framework" as a gulda to critical procaasos. 

Even though the assessment of the Xube** as a guide to critical processes 
was explicitly stated in advance, the descriptions supplied of the consid* 
ered environments related very little to this aspect. Still it is clear that a 
properly functioning ''Cubic Framework" should be capable of being used 
as a guide or checklist for individuals involved in specific aspects of a 
courseware development process. The reason that this type of '*gukle'* 
function did not emerge clearly couki be the above-mentioned remark that 
the "Cube" was still not referred to as a whole of colls (and subcells). but 
only as its Individual elements according the three separate dimensions. 

in order to better fulfil a ''guide to critical processes" function, we have 
elaborated an approach using a fk)W chart-structure, whk:h will be pre- 
sented below. It was a first attempt of showing how the "Cube" coukI be 
used as a guide to critical processes during the design and devek>pm6nt 
activity. 

Within the "Cubic Framework'* the major terms can be subdivided Into 
more specific terms. In the example shown in the fk)w chart, the Activity- 
term ''Specify'* (which is part of the overall activity-term "Develop'') is sub- 
divided into more detailed terms in order to be able to identify a specifk: 
sequence of these (sub-)activities. All of this can be visualized by a flow 
chart as is presented in Figure 2. 



28 



Figure 2.: Flow-chirt •xampto: "Sptclfy^sliM. 
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As is obvious the flow chart gives an entry into the framework in more than 
a descriptive fashion: it tries to synthesize a ssquence of activities and 
thus to function as a guide (checklist) to critical processes for devetopers. 
This approach can be folk>wed at any specific entry point in the develop- 
ment-process« If the fk)w chart could be specified with enough detail (i.e., 
provides enough information), it can make the user aware of any activity 
(and actor/resource) and its successor(8) in the sequence of activities per- 
tinent to a specif to critteal process, and of problems that can occur in that 
sequence. 

The fk)W-chart approach has the potential of being an optimal elaboratton 
of the framework for gukJing purposes because of its representatbn of se- 
quences within processes. However to increase future accessabli!ty, we 
would like to suggest to implement the Xubic Framework** according to a 
hypertext representatton form, tf the '*Cubk; Framework** wouki be 
elaborated in such a form, it shoukJ be more generally recognized as 
appropriate for a European TSRM". (For an elaboration of the hypertext 
approach refer to Section S.) 



4.4. The "Cubic Framework" ae a reference to critical leauee. 

An important aspect of the "Cubto Framework" as a ''Learning System Ref* 
erence Model** is its potential to serve as a reference to the major issues 
that are unJ will be defined in the context of DELTA. As we have seen in 
Section 2, some conclusions coukl already b)e made on several of these 
issues such as: 

«• Systems interopernbility (standardization); 

- Learning support systems; 

- Pedagogical strategies; 

- Anticipating the European context. 
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However, the use of these and future outcomes of studies on critical 
Issues depends on the accessibility of the results and recomnriendations. 
In order to be accessed, references to these Issues should be a part of a 
hypertext approach, as will be proposed In the next section. Specific 
attention should be made to the format In which material relative to these 
Issues will be available and also If suggestions on this format should be 
prescriptive or descriptive. About this last aspect a discussion should bp 
held In the ''European Learning Technology Association** which will be 
founded In 1989. 



4.5. Conclusion. 

The "Cubic Frameworl<'* has the potential to serve the three purposes Indl* 
cated for its assessment. However, it should be noted that an elaboration 
should take place relative to the following aspects: 

* the terms used need more specific definitions; 

* there should be a clear entry Into the framework according the type of 
development environment that Is considered; 

for optimal functioning the ''Cubic Framework" should be Implemented In 
a hypertext system; 

* for varbus specifb aspects of the courseware devetopment process flow 
charts shoukJ be elaborated; 

« the conclustons of current and future research on klentif led issues shouki 
be In a specific format to be Included In the hypertext-based "Cubic 
Framework". 
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5: FURTHER RESEARCH. 



Introd Jction. 

In this section an attempt Is made to suggest the further development of a 
detailed elaboration of the "Cubic Framework** for a **Learnlng System Ref** 
erence Model** as Is supposed to tal<e place In the next phase of the 
DELTA progi'amme of the European Community. 

First a short overview Is given of this phase, which is called the 
'*Exploratory Action** of DELTA. Then some suggestions from the previous 
sections are elaborated and an outline of a context Is given for an optimal** 
izatlon of further elaborations. 



The DELTA Exploratory Action. 

The projects under the DELTA Exploratory Action are divided Into five so- 
called **Action Llnes^ These action lines are: 

Action Line 1 ; Learning Svstems Research. 

The major purpose of Actbn Line 1 1s to gather and disseminate the out« 
comes of the other four action lines. It Is the catalyst for exchange of In- 
formation and more specifically for harmonization and Integration activities; 

Action Line 2: Collaborative Development of Advanced Learning 
Tech'nologv. 

The aim of this action line is to complement the already existing efforts of 
the IT&T Indust7 In .)rderto achieve a European design of systems and 
equipment by collaboration. The results of this action line should be a 
Tortable Educational Tool Environment" ("PETE"); 
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Action Line 3: Testing and Validation of Communications and of SQFJ 
(Satellite based Open Facllitv for Testing^ 

This action line Is intended for the specification of communications facilities 
to "serve the learning community" In Europe. In a special place here Is the 
use of a satellite as a test-bed for Important aspects of the other action 
lines; 

Action Line 4: Interoperability 

Central In this action line Is: 

- to ensure work on standardization; 

• to identify relevant work In areas that are not currently considered; 

Action Line 5: Promotion of Favourable Conditions 
This action line will be focused on aspects In the social, fiscal and regula- 
tory environment in which DELTA Is intended to be embedded. 

As Its name ("Learning Systems Research") Indicates, Actton Line 1 In- 
cludes the successive action i.n the LSRM and the elaboration of the 
"Cubte Framework". However aspects of this L'arning System Reference 
Model can be found In any action line. 

In reflection, the DELTA Pre-Pltot project can be said to have focused on 
providing Input for Action Line 1 . In the context of the start of this action 
line, some suggestions are elaborated in the next paragraph. 
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5.3. Suggestions for slaborstion of ths LSRM. 



5.3.1. LSRM Implsmsntsd on a hyportsxt systsm 

A firsts more detailod elaix>ratlon of the Xubic Framework** Is the identifi- 
cation of specific flow charts aocordlrKi to sub-parts of the '*Cubic Frame- 
worli". A short example of this approach was given In Section 4. But it 
should be noted that a two-dimensional flow chart has its limitations* al- 
though with more detailed Information about the components such as the 
matching of actors and resources to specific activities* the identified critical 
Issues and a description of all these aspects, should be reachable. A hy- 
pertext representation of the '*Cubic Framework** can serve these purposes 
and has as surplus the possibility of constant updating and refining. A ma- 
jor condition for this approach is of course a clear structure, but also a 
wkle area network system for storage and access. 

Structure 

A basic structure for a hypertext representation of the Xubic Framework** 
according e flow chart elaboration oouki be: 

- Start screen: an oven/lew of the major terms of the activities-dimension 
and the first major chok:e: selection of the presentatton of a spedfk; fk)w 
chart; 

- Second level: optton to choose (the top-level of) a specific activity of that 
flow chart; 

- Third level its place In the sequence, on basis of **input fromToutput to**; 

- Fourth level: references to the (top level of the representation of) possible 
actors and resources; 

- Fifth level: a description of the activity and of its Input an J uutput 
(including references); 

- Sixth level: references to (the top level of the representation of) specific 
problems (Issues). 

Working out this approach In the next phase of DELTA (how or if this could 
be done was still not clear at the end of the DELTA Pre-Pllot project) would 
need a consklerable cooperation with and input from workers throughout 
DELrA as well as the capability of almost constant updating. However, as 
noted earlier, if it can be developed this way, the framework will be more 
generally recognized as appropriate for a European **LSRM**. 
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Limitation. 

As is clear, the range of activities (and actors / resources) included in the 
Framework is a very wide one. Therefore it would be useful to provide a 
shell through which the implemented '*Cubic Framework** can be accessed 
according the type of development environment that is considered. This 
shell should limit the width of the area or the depth of the area that is ac- 
cessed. The aim of this limitation is to keep the overview manageable 
within the larger scope of the **Cubic Framework^ 

The limitation is suggested in accordance with differences that can be 
identified in the levels of organization of a learning materials devetopment 
project**. 

We see three levels of organizatton, related also to the functton the 
courseware shoukJ have within a specifk: currksulum. These levels are: 

1 : ExperimMtal. One team (e.g. an learning-material-devetoper, a sub* 
Ject-matter-expert and an instructbnal-expert), Is making some 
courseware . This courseware can be experimental, but after an ex* 
ptoratory phase the devek>ped courseware can also have a supple- 
mentary function within a curriculum. At this level there is hardly any di« 
rect influence from **outskie** (government, sodetyt manufacturer or 
commercial publisher). 

2: Organized. After a group has worked at an experimental level and its 
courseware is used to supplement a currksulum, there may arise a 
need to formalize the devetopment-process. The indivkluals making up 
the team are grouped within the organizatton (e.g., a group of learning- 
material-devetopers, agroupof subject-matter-experts). At this level 
the influence from **outskje** is growing, but still is r)ot formalized. The 
developed courseware evolves from supplementary to, in some cases, 
complementary (necessary for specif k: curricula) and gets at least a 
more formal plac^ within specif currk^ula. 
3: Institutionalized, If the process continues, the courseware-developing 
activity exceeds the resourc n of one organization. The different 
groups involved in the development are becoming instituttonalized and 
this means that the development-process becomes formalized.ln addi- 
tion the influence from **outside** becomes formalized. The courseware 
madG by these institutions can be supplementary, but in general its 
goal will be to be complementary or even to be **substitutive** 
(enck)slng the total curricuium(-part)). 
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5.4. Th« LSRM and th» DELTA Exploratory Action. 

First attempt for a further elaboration of the "Cubic Frameworit" as a 
Learning Systeno Reference Model is given above. However, this attempt 
needs a critical refinement of the suggested shell and of the suggested hy- 
pertext structure. Both refinements should preferably take place in the 
context of the ideas which have developed since the launch of the Ex- 
ploratory Action of DELTA. 

5.4.1. ELTA. 

One of these ideas is a major outoome of Task Force 1 called "ELTA": Eu- 
ropean Learning Technology Assodatfon. The major aim of this associa- 
tfon is to establish a constant visibility for <ts members of the progress of 
projects within DELTA and related areas. One way to f uKII this aim is by an 
ELTA-bulletin and ELTA-Seminars in whk:h outcomes of the actfon lines 
will be announced, described and discussed. Furthermore "ELTA" will 
organise a large DELTA-conference in October 1990. 

ELTA can be seen as the common platform in whk:h the several aspects of 
an LSRM can be discussed. 



5.4.2. Implttmentatlon of the LSRM. 

It still was not clear at the moment that this paper was written, if the elabo- 
rated "Cubic Framework" will be accepted by the European Commissbn 
as the LSRM and if it also will be implemented in the Exploratory Actton of 
DELTA through a wide area network to be accessible to any DELTA-in- 
volved actor. 
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However, the current elaborated Xubic Framework" for a Learning 
System Reference Model hae dratted an outline for the range of research 
of the whole 8r88 of DELTA and should thorsforG actiVGly ssfvo as an 
input for the activities of Action Line 1 in order to also have Impact on the 
other action lines The decision atMut continuation of the work on an 
"LSRM" Is to be taken after a so-called LSRM-workshop, to be organized 
by the European Community In 1989. At this workshop the results of the 
DELTA Pre Pitot project concerning "LSRk " will be evaluated and 
depending on the results of that discusston, a declston will be made It the 
actual work on the "Cubk: Framework" or and DELTA LSRM more 
generally will be continued. 



Conclusions. 

What has become clear Is that the outcomes of the DELTA Exptoratory 
Actk>n need a more refinement of at least the foltowing aspects: 

- Definltton of the used terms/lerminotogy (e.g. Actors and Activities); 

- Continued standardlzatk)n towards one termlnokigy; 

- Agreements about standardization of equipment: Hardware & Software 
(tools) and Learning Support systems; 

- Agreements about the directtons to addrass spedfk; Issues (such as: 
pedagogical strategies, applk»tlon of kinds of artificial intelligence, 
cultural (and subcultural) differences). 

• Agreements that for optimal functtoning, the "Cubic Framework" should 
be implemented In a hypertext system; 

- Clear descriptton (towards standardization) of spedfk) sub-areas of the 
courseware developing process, e.g. by elaborating the fk>w charts ap- 
proach; 

• Agreement that there should be a clear entry into the framework 
according the kind of development environment that is oonsklered; 

- Specification of a specific format for the conclusions of cun'ent and future 
research on identified issues for effective Incluston in the hypertext-based 
"Cubic Framework". 
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There are at least two possible ways to reach this tuning: 

- to have a further elaboration of the LSRM*ooncept towards a real-existing 
system as part of Action Line 1 (action 1.0). 

- to strengthen the concept of an "European Learning Technology As* 
sedation'' (ELTA) as a (Ihe") leading association In Europe and the 
place of exchange In order to have the above-mentioned aspects solved 
In presumably generally accepted approaches. 

It is for the Commission of the European Community to make the choice 
on continuation. However, we can make a statement about the impact of 
this choice. We believe that only through a choice of both of the above- 
mentioned ways to reach this refinement there will be continuity of the 
original ideas as were stated In the Initial Studies (Integration, Standard- 
ization and a European wide cooperation towards one Learning System 
Reference Model, see: Initial Studies, 1687). A positive secondary effect is 
that there will be a contint lity In all the money and energy of people that 
was Invested In the DELTA Pre-Pllot project. Finally, it includes also the 
setting of the "Main-phase" of the DELTA-programme (1992-1996). In this 
phase the results of the harmonization and integration should be broadly 
explored and should lead evpntually to European-wide applicable results. 
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Appendix: Detailed oven/iew of the terms of the Xublc Framework'' 



1 



DIMENSION: ACTIVITIES 

A. MANAGE 

GENERAL 

find rasourcas 

control costs 

develop own skills 

review 

improve 

run project 

plan project 

set up project 

negotiate contact 

commission (external) 

producer 

negotiate copyrights 



MARKETING 
set targets 
Identify market 
decide on actton 
investigate market 
define product 
stimulate use of learning 
material 

mediate with other 
organisations 
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Apporniix: D^tallBd overvtew of the terms of tho ''Cubic FramBWork'' 



SALES 

promotd & advortise 
visit (potential) clients 
handle order 
supply 
Invoice 



SUPPORT 

engage staff 
train the tutor 
support after sales 



B. FACItlTATE 

subsidise 
sponsor 
legislate 
promote 



a STANDARDISE 

define application 
standards 

define quality standards 
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AppBndix: Detailed overview of the terms of the ""Cubic Framework 



D. SUPPLY 

supply delivery 
systems 
supply tools 



E* DEVELOP 

SPECIFY 

study feasibility 
define Learning 
Environment 

analyse target population 
define alms 
analyse needs 
define prerequisites 
define curriculum 
define delivery system 
define objectives 
define instructional 
strategies 
define presentation 
media 

define assessment criteria 
define assessment questions 
define layout 
define learner monitor 
define security measures 
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Appendix: Detailed oven/lew of the terms of the "Cubic Framework*" 



DESIGN 

Investigate existing 
learning material 
design new learning 
material 

analyse subject matter 
design structure of 
learning material 
write script 
(AV/CBT/Kit/Books) 
design layout 
design learner monitor 
design security measures 
design package 
develop prototype 



PRODUCE 

write course text 

record & edit Audk)-Vlsual material 

code software 

make (experimentation) kit 

produce graphtes 

test & debug modules 

write manual text 

write (technical) documentation 

transferAranslate 



EVALUATE 

check with client during development 
evaluate (formative) during devek)pment 
perform field test 
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Appendix: Detailed oven/lew of the terne of the '*Cublc Framework** 



ASSEMBLE 

intograta package 
test integrated package 
reproduce package 
store material 



F. Distribute 



ERiC 



administrate 

broadcast, downtoad. or mall learning material 
charge / lnvok:e 



Q. USE 



counsel 

select learning material 

coach /tutor 

learn 

monitor 



H. EVALUATE 



maintain 

evaluate (summativo) learning material 
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Appendix: DQtailed ov&Mbw of the terms of the ''Cubic Framework** 



DIMENSION: ACTORS 

A* Government 

legislator / ministery of education 
validating body 
visiting committe 



B. SOCIETY 

professbnal body 

sponsor 

examiner 

standardisation board 
learner association 



a EXPERT / ACADEMIC BODY 

subject matter specialist 
professional institute 
librarian (academic) 
evaluator (external) 



D. MANUFACTURER 

mari<dting staff 
sale'3 staff 
support staff 
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Appendix: Detailed overview of ttie terms of the 'Cubic Framework" 
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E DEVELOPER 

PROJECT MANAGEMENT 
project manager 



SPECIFICATION STAFF 
curriculum designer 
technical specifier 



DESIGN STAFP 

Instrudbnal expert 
subject matter expert 



PRODUCTION STAFF 

- producer (AV / CBT / Kit / Book) 
production manager 

writer 
typesetter 

- editor (AV/ Text) 
graphic designer (illustrator) 
user interface d^v<)loper 
programmer 



ASSEMBLING STAFF 
programmer 
translator 
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Appendix: Detailed overview of the terms of the ""Cubic Framework** 



EVALUATION STAFF 



SERVICES STAFF 

librarian (of Learning Material) 
computing services staff 



F. PUBLISHER /SERVICER 

marketing staff 
sales staff 

implementation manager 
warehouse manager 
administrator 

telecommunications Operator 
legal expert 
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Appendix: Detailed overview of the terms of the Xubh Framework^* 
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G. CLIENT 



spokesman 
purchasdr 

Implementation manager 

mentor 

tutor 

learner (actual) 
learner (potential) 



DIMENSION: RESOURCES 

A. HUMAN SKILLS & UNDERSTANDING 

Human Skills & Understanding can be related to: 

Education 
Subject matter 
Management 
Commercial 
Presentation media 
Information technology 



B. METHODOLOGIES 

Standards 
Guidelines 
Procedures 
Techniques 
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Appendix: DetwM ovqivIbw of the terms of the Vubic Framework'* 



C. SOFTWARE 

Systom 
Gonoric 

Dedicated (to Learning) 

D. HARDWARE 

Processing 
Storage 

Communication 
Input 

Presentation 
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